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DETAILED ACTION 

Priority 

1 . Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), which 
papers have been placed of record in the file. 

Information Disclosure Statement 

2. The references listed on the Information Disclosure Statement filed on 
September 25, 2003 have been considered by examiner; see attached PTO-1449. 

Specification 

3. The abstract of the disclosure is objected to because of legal phraseology such 
as "means" and "said". Correction is required. See MPEP § 608.01(b). 

Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

5. Claims 16-23 are rejected under 35 U.S.C. 101 , because the claimed invention is 
directed to non-statutory matter subject matter. 
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Claims 16-23 recite "A program", which is nonspecific and lack utility. The 
computer program product can be reasonably interpreted to be, software per se. That 
is, the scope of the presented claimed a computer program that can range from paper 
on which the program is written, to a program simply contemplated and memorized by a 
person. Claims 16-23 neither include any computer hardware components nor 
positively recite that the cited computer program are stored on a computer medium that 
can be read by a machine. As such, claims 16-23 are directed toward software per se, 
which is non-functional descriptive material and non-statutory. 



Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

(b) the invention was patented or described in a printed publication in this or a foreign country or in 
public use or on sale in this country, more than one year prior to the date of application for patent in 
the United States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



5. Claims 1-4, 7, 9-12, 15-19, 22-23, 24-27 and 31 are rejected under 35 
U.S.C. 102(b) as being anticipated by Okazaki et al. (US Patent: 5,666,555). 
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As to claim 1 , Okazaki discloses a multichannel information processing device 
(i.e. the multiple video signal coming into the system like VTR and LD) (see Fig. 1, Col. 
2, Lines 41-57) wherein a plurality of video images are displayed simultaneously on a 
display device (i.e. multiple windows on the display screen of the computer displaying 
separate video signals) (see Fig. 1 , Col. 2, Lines 58-68), comprising: 

video image information control means (103 CPU) for acquiring information for 
said plurality of video images, and for deciding video image position information relating 
to display position on a display device (108) for said plurality of video images (i.e. 
images coming from 101 image reproduction apparatus) and outputting said information 
for a plurality of video images based on said video image position information (i.e. since 
the CPU process the video image information and assign these information on to a 
plurality of windows on the Bit Map Display 108, it must assign a position value for each 
of the windows in order to display the each of the video images correctly) (see Fig. 1 , 
Col. 2, Lines 58-68); 

cursor position control means (103 CPU) for calculating cursor position 
information based on cursor instructions information input via an input device (i.e. key 
board 111, pointing device 105) and generating and outputting cursor image information 
based on said cursor position information (i.e. since the cursor is generated by the IOP 
104 and controlled and processed by the CPU 103 when user input devices 105 is 
activated the cursor must be calculated by the CPU in order to allow the position value 
to be properly accessed and utilized for the selection of the various windows) (see Fig. 
1, Col. 3, Lines 5-12); 
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display image generating means (106) for synthesizing information for the 
plurality of video images output by said video image display control means (103) and 
cursor image information output by cursor position control means (103) and displaying 
the same on said display device (i.e. since the multiple video images are display on the 
Bit Map Display 108 in forms of multiple windows and each of the window can be 
selected by the cursor to activate its audio content, the video images are successfully 
synthesized and displayed) (see Fig 1, Col 2, Lines 41-68) ; 

distance information generating means (CPU 103) for calculating distance 
between display positions of said plurality of video images and cursor display position 
based on video image position information for said plurality of video images and cursor 
position information calculated by said cursor position information control means, and 
generating distance information (i.e. since each of the windows are selectable by the 
cursor to activate its audio content the distance is therefore one or zero where if the 
cursor was found to be within the window it is zero and the audio is outputted and when 
the cursor is outside the window then the distance is 1 and no audio is outputted) (see 
Fig. 1, Col. 3, Lines 5-12); 

and audio output control means (i.e. Audio Selector 102 ) for deciding volume of 
audio data for said plurality of video images based on distance information generated by 
said distance information generating means, and outputting audio data to an output 
device(i.e. since when the distance is zero, where the cursor is found to be inside the 
window, then the corresponding audio of the video in the window is outputted at a 
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preset volume, and when the distance is one, where the cursor is not inside the window 
the volume is zero and the audio is not outputted) (see Fig. 1, Col. 3, Lines 5-12). 

As to claim 16, Okazaki teaches a program for executing on a computer a 
multichannel information processing method for displaying a plurality of video images 
simultaneously on a display device (i.e. the multiple windows on the screen each having 
video images displayed coming from the disk which store image data and audio data) 
(see Fig. 4, Col. 4, Lines 14-41), such method comprising: 

a step for deciding display position on display device for said video images to be 
displayed (i.e. since the computer CPU 103 process the video image data and output 
the data on to a plurality of display windows it necessary decide the position that the 
video image takes up in order to properly display it on bit map display 108) (see Fig. 4, 
Col. 4, Lines 14-26); 

a step for outputting information for said plurality of video images based on 
decided display position (i.e. the CPU 103 sent the video images into the various 
windows on the bitmap display 108 based on the display address assigned) (see Fig. 4, 
Col. 4, Lines 14-34); 

a step for accepting cursor instructions information input from said input device 
(pointing device 105) (i.e. the pointing device 105 inputting the pointer information via 
the IOP 104 to the CPU 103) (see Fig. 4, Col. 4, Lines 47-58); 
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a step for calculating cursor position information for displaying cursor based on 
said cursor instructions information (i.e. the location of the pointer on the bitmap display 
108 must be calculated to update the operation of the pointing device 105) (see Fig. 4, 
Col. 4, Lines 47-58); 

a step for generating cursor image information based on said cursor position 
information (i.e. the image of the pointer on the bitmap display 108 is outputted 
therefore it must be generated after the updates is made on the operations of the 
pointing device 105) (see Fig. 4, Col. 4, Lines 47-58); 

a step for synthesizing information for said plurality of video images and said 
cursor image information, generating a display image, and displaying the same on said 
display device (i.e. the outputted screen synthesized image data is output as an image 
signal to the bit map display 108 via the D/A converter 305, and since the pointer is 
present is must be synthesized also) (see Fig. 4, Col. 4, Lines 20-27); 

a step for calculating distance between display position of said plurality of video 
images and display position of said cursor and generating distance information (i.e. 
since the distance of the selected windows is zero (i.e. selection) for turning on, when 
the cursor is outside the parameter of a window it has a distance of one (i.e. non- 
selection), which is calculated based upon this position of the windows and the current 
pointer position) (see Fig. 4, Col. 4, Lines 46-63); 

and a step for deciding volume for audio for said plurality of video images based 
on said distance information and outputting to an audio output device accordingly (i.e. 
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the volume of the audio of the video is selected based upon the cursor, where the 
window that is selected has the nominal value and the non-selected has a muted 
volume) (see Fig. 4, Col. 4, Lines 46-63). 

As to claim 17, note the discussion of claim 16 above, claim 17 differs from claim 
16 only in the limitation of a step for setting volume for audio data for the video image 
having the smallest distance value among the distance information generated with 
regard to said plurality of video images at a predetermined sound volume Va, and 
setting volume for audio for the other video images at predetermined sound volume Vb, 
which is lower than volume Va, and outputting to an audio output device accordingly. 

Okazaki teaches a step for setting volume for audio data for the video image 
having the smallest distance value among the distance information generated with 
regard to said plurality of video images at a predetermined sound volume Va (i.e. the 
nominal value original set), and setting volume for audio for the other video images at 
predetermined sound volume Vb (i.e. the muted volume of zero), which is lower than 
volume Va, and outputting to an audio output device accordingly (i.e. since the non- 
selected windows have a greater distance than the selected window they have a muted 
zero volume compared to the normal volume of the selected window which is outputted 
to the speaker 109) (see Fig. 4, Col. 4, Lines 46-63). 



Application/Control Number: 10/669,508 Page 9 

Art Unit: 2629 

As to claim 18, note the discussion of claim 16 above, claim 18 differs from claim 
16 only in the limitation "setting volume for audio data for said plurality of video images 
so as to be in inverse proportion to distance value in distance information". 

Okazaki teaches setting volume for audio data for said plurality of video images 
so as to be in inverse proportion to distance value in distance information (i.e. since the 
non-selected windows have a value of one than the selected window's value of zero 
they have a muted zero volume compared to the normal volume of the selected window 
which is inversely proportional based on distance) (see Fig. 4, Col. 4, Lines 46-63). 

As to claim 19, note the discussion of claim 16 above, claim 19 differs form claim 
16 only in the limitation of: a step for generating direction information relating to 
direction of display position for each video image as seen from cursor display position; 
and a step for making output to said audio output device so that audio data 
corresponding to said plurality of video images is positioned at acoustic image positions 
in the sound space of said audio output device in accordance with said distance 
information and said direction information. 

Okazaki teaches a step for generating direction information relating to direction of 
display position for each video image as seen from cursor display position (i.e. since the 
pointer value that is represented on the display 108 is in two direction x and y the video 
also has those direction since that is how a bitmap display receive the data and display 
it on screen) (see Fig. 4, Col. 4, Lines 46-63); 
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and a step for making output to said audio output device so that audio data 
corresponding to said plurality of video images is positioned at acoustic image positions 
in the sound space of said audio output device (speaker 108) in accordance with said 
distance information and said direction information (i.e. since the output the center 
selected audio data based on the pointer selection, it is in the sound space of the 
speaker as the data values that are outputted there) (see Fig. 4, Col. 4, Lines 46-63). 

As to claim 23, note the discussion of claim 16 above, claim 23 is broader in 
scope than claim 16 as the limitation "a prescribed algorithm" can be any method 
applying the images, and is rejected on the same ground. 

As to claim 2, Okazaki teaches a multichannel information processing device 
according to claim 1, wherein said audio output control means (102) sets volume for 
audio data for the video image having the smallest distance value among the distance 
information (i.e. since each of the windows are selectable by the cursor to activate its 
audio content the distance is therefore one or zero where if the cursor was found to be 
within the window it is zero and the audio is outputted and when the cursor is outside 
the window then the distance is 1 and no audio is outputted) generated by said 
distance information generating means (103) at a predetermined sound volume Va (i.e. 
the original volume that is designated by the input system), and sets volume for audio 
corresponding to the other video images at predetermined sound volume Vb (i.e. zero 
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volume, when the non-selected windows are muted), which is lower than said volume 
Va. (see Fig. 1, Col. 3, Lines 5-12) 

As to claim 3, Okazaki teaches a multichannel information processing device 
according to claim 1, wherein said audio output control means (102) sets volume for 
audio data for said plurality of video images so as to be in inverse proportion to distance 
values generated by said distance information generating means (i.e. since each of the 
windows are selectable by the cursor to activate its audio content the distance is 
therefore one or zero where if the cursor was found to be within the window it is zero 
and the audio is outputted and when the cursor is outside the window then the distance 
is 1 and no audio is outputted. In this way the volume is inverse proportion to the 
distance value generated) (see Fig.1, Col. 3, Lines 5-12). 

As to claim 4, Okazaki teaches a multichannel information processing device 
according to claim 1 , wherein distance information generated by said distance 
information generating means (103) includes direction information relating to direction of 
video image display position as seen from cursor display position (i.e. since the 
windows on the screen have a two dimensional outlay x and y by which an input cursor 
is placed, the direction are accounted for during the calculation for the positional 
information of the window as compared to the cursor), and said audio output control 
means (102) makes output to an audio output device based on said distance 
information, so that audio data for said plurality of video images is positioned in the 
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sound space formed by said audio output device (i.e. since each of the windows are 
selectable by the cursor to activate its audio content the distance is therefore one or 
zero where if the cursor was found to be within the window it is zero and the audio is 
outputted and when the cursor is outside the window then the distance is 1 and no 
audio is outputted. Also the audio signal of any windows must be in the sound space of 
the speaker 109 when it is outputted since it is the only audio output means in the 
system) (see Fig.1, Col. 3, Lines 5-12). 

As to claim 7, Okazaki teaches multichannel information processing device 
according to claim 1 , further including video image selecting means for selecting, based 
on a prescribed algorithm (i.e. since the video image reside in the windows on the 
screen and the window is selected by the cursor, an algorithm is used to determine the 
window that is selected and therefore the video that is selected), a specified video 
image from among a plurality of video images displayed on said display device, wherein 
said audio output control means outputs to an audio output device audio data for the 
video image selected by said video image selecting means (i.e. since the audio and 
video signal are presented together when the window is selected, both the audio and 
video are output upon selection) (see Fig. 1, Col. 3, Lines 5-12). 

As to claims 9-12 and 15, these claims differ from claims 1-4 and 7 only in that 
claim 1-4 and 7 are apparatus claims, whereas claim 9-12 and 15 are method claims. 
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Thus, claims 9-12 and 15 are analyzed as previously discussed with respect to claims 
1-4 and 7 above. 



As to claims 24-27, these claims differ from claims 16-19 only in that claim 16-19 
are program claims, whereas claim 24-27 are media storage claims. Thus, claims 24- 
27 are analyzed as previously discussed with respect to claims 16-19 above. 

As to claim 31, this claim differs from claim 23 only in that claim 23 is a program 
claim, whereas claim 31 is a media storage claim. Thus, claims 31 is analyzed as 
previously discussed with respect to claim 23 above. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 20-21 , 5, 13, and 28-29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Okazaki in view of Yamagami (U.S. Patent 6,334,025). 
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As to claim 20, note the discussion of claim 16 above, Okazaki does not teach a 
step for voice-recognizing words included in audio data for said plurality of video 
images; a step for converting voice-recognized words into character data and outputting 
the same; 

Yamagami teaches a step for voice-recognizing words included in audio data for 
said plurality of video images (i.e. the CPU 13 execute the audio recognition causing the 
result to be displayed in the display section 402); a step for converting voice-recognized 
words into character data and outputting the same (i.e. the CPU 13 execute the audio 
recognition causing the result to be displayed in the display section 402) (see Fig. 4, 9, 
Col. 10, Lines 5-23, Col. 12, Line 60 - Col. 13, Line 10); 

Therefore, it would have been obvious for one of ordinary skill in the art at the 
time the invention was made to combine the voice recognition capability of Yamagami 
to the multi-window display of Okazaki in order to allow a more efficient storage of the 
annotation of audio and video data (Yamagami, Col. 1 , Lines 65-68). 

As to claim 21 , note the discussion of claim 16 and claim 20 above, claim 21 
differs from claim 20 only in the addition of two addition steps: 

a step for calculating distance between position information for said plurality of 
video images and said cursor position information and generating distance information; 

a step for selecting a specified video image from among the plurality of video 
images based on said distance information and outputting audio data for this video 
image to an audio output device; 
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Okazaki teaches a step for calculating distance between display position of said 
plurality of video images and display position of said cursor and generating distance 
information (i.e. since the distance of the selected windows is zero (i.e. selection) for 
turning on, when the cursor is outside the parameter of a window it has a distance of 
one (i.e. non-selection), which is calculated based upon this position of the windows and 
the current pointer position) (see Fig. 4, Col. 4, Lines 46-63); a step for selecting a 
specified video image from among the plurality of video images based on said distance 
information and outputting audio data for this video image to an audio output device (i.e. 
the audio data and the video data of the selected window is output to the bitmap display 
108 and the speaker 109 based on the distance zero which is when the cursor is 
actually in the window selected) (see Fig. 4, Col. 4, Lines 46-63).; 

As to claim 5, Okazaki teaches a multichannel information processing device 
according to claim 1 , but does not teach voice data recognition means for recognizing 
words included in audio data for said plurality of video images, and character 
information display means for converting words recognized by said voice data 
recognition means into character data and displaying the same on said display device. 

Yamagami teaches voice data recognition means (13 CPU) for recognizing 
words included in audio data for said plurality of video images, and character 
information display means (13 CPU) for converting words recognized by said voice data 
recognition means into character data and displaying the same on said display device 
(i.e. the CPU 13 execute the audio recognition causing the result to be displayed in the 
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display section 402) (see Fig. 4, 9, Col. 10, Lines 5-23, Col. 12, Line 60 - Col. 13, Line 
10). 

Therefore, it would have been obvious for one of ordinary skill in the art at the 
time the invention was made to combine the voice recognition capability of Yamagami 
to the multi-window display of Okazaki. 

As to claim 1 3, this claim differs from claim 5 only in that claim 5 is an apparatus 
claim, whereas claim 13 is a method claim. Thus, claims 13 is analyzed as previously 
discussed with respect to claim 5 above. 

As to claims 28-29, these claims differ from claims 20-21 only in that claims 20- 
21 are program claims, whereas claims 28-29 are media storage claims. Thus, claims 
28-29 are analyzed as previously discussed with respect to claims 20-21 above. 

8. Claims 22, 6, 14, and 30 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Okazaki in view of Yamagami as applied to claims 20-21 , 5, 13, and 
28-29 above, and further in view of Hilpert, Jr. et al. (U.S. Patent 6,469,712) 

As for claim 22, note the discussion of claim 20 above, Yamagami does not 
explicitly teach a step for connecting to the Internet; a step for searching for related web 
sites on the Internet using a voice-recognized word as keyword; 
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Hilpert teaches a step for connecting to the lnternet(i.e. Internet access 
program); a step for searching for related web sites (i.e. Net Search) on the Internet 
using key word (i.e. since the web browser is an extension to the conventional process 
data capability of an individual personal computer, it is natural to use the web search 
capability to enhance the operation of the computer) (Col. 3, Line 50 - Col. 4, Line 50). 

Therefore, it would have been obvious for one of ordinary skill in the art at the 
time the invention was made to combine the web searching capability of Hilpert to the 
text recognition design of Yamagami, (i.e. having the text recognition process to be 
further enhanced by use of Internet search for more detailed information) in order to 
provide the user additional interactions with the images displayed to assist visually 
impaired users (Hilpert Col. 1 , Lines 50-66). 

As to claim 6, note the discussion of Claim 5, Yamagami does not explicitly teach 
Internet connection means, web site search means for searching for related web sites 
on the Internet and web site display means for displaying on said display device a web 
site found by said web site search means. 

Hilpert teaches Internet connection means (i.e. Internet access program), web 
site search means (i.e. Net Search) for searching for related web sites on the Internet 
and web site display means for displaying on said display device a web site found by 
said web site search means (i.e. since the web browser is an extension to the 
conventional process data capability of an individual personal computer, it is natural to 
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use the web search capability to enhance the operation of the computer) (Col. 3, Line 
50 - Col. 4, Line 50). 

Therefore, it would have been obvious for one of ordinary skill in the art at the 
time the invention was made to combine the web searching capability of Hilpert to the 
text recognition design of Yamagami (i.e. having the text recognition process to be 
further enhanced by use of Internet search for more detailed information). 

As to claim 14, this claim differs from claim 6 only in that claim 6 is an apparatus 
claim, whereas claim 14 is a method claim. Thus, claims 14 is analyzed as previously 
discussed with respect to claim 6 above. 

As to claim 30, this claim differs from claim 22 only in that claim 22 is a program 
claim, whereas claim 30 is a media storage claim. Thus, claims 29 is analyzed as 
previously discussed with respect to claim 6 above. 

9. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Okazaki 
in view of Tarabella (U.S. Patent 5,796,945). 

As to claim 8, note the discussion of Claim 7, Okazaki does not explicitly teach 
video image selecting means switches to a different video image for selection whenever 
a prescribed length of time has passed. 
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Tarabella teaches video image selecting means switches to a different video 
image for selection whenever a prescribed length of time has passed (i.e. the video clip 
that is capable of being displayed can be controlled for the length of time that it is to be 
displayed before a change is to take place (see Col. 5, Lines 1-53). 

Therefore, it would have been obvious for one of ordinary skill in the art at the 
time the invention was made to combine the time based pre-set selection capability of 
Tarabella to the image selection system of Okazaki, in order to make the computer 
display more productive during idle time (see Tarabella, Col. 2, Lines 5-13). 

Conclusion 

Duhaul et al. (U.S. Patent 6,1 18,493) is cited to teach a multiple video display 
system. Kaneko (U.S. Patent 5,273,418) and Park and et al. (U.S. Patent 6,590,618) 
are cited to teach multiple video display system having multiple sound output and 
control. 



Inquiry 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Calvin Ma whose telephone number is (571) 270-1713. 
The examiner can normally be reached on Monday - Friday 7:30 - 5:00 EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chanh Nguyen can be reached on (571) 272-7772. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Calvin Ma 
August 1 , 2007 




CHANH D. NGUYEN 
SUPERVISORY PATENT EXAMINER 



